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Synthetic  Flight  Training  Revisited 

BEVERLY  H.  WiLLIGES,  STANLEY  N.  ROSCOE,  and  ROBERT  C.  WILLIGES, 
University  of  Illinois  at  Urbana-Champaign 

Critical  issues  in  the  development  and  use  of  synthetic  flight  trainers  are 
reviewed.  Degree  of  simulation  and  fidelity  of  simulation  are  discussed  as  key 
design  considerations.  Problems  of  measurement  of  original  learning,  transfer,  and 
retention  are  present  d.  Both  transfer  effectiveness  and  cost  effectiveness  are 
described  as  critical  factors  in  the  evaluation  of  flight  trainers.  Recent  training 
innovations,  such  as  automatically  adaptive  training,  computer-assisted  instruction, 
cross-adaptive  measurement  of  residual  attention,  computer  graphics,  incremental 
transfer  effectiveness  measurement,  and  response  surface  methodology,  ore  discussed 
as  potential  techniques  for  improving  synthetic  flight  training.  It  was  concluded 
that  broader  application  of  simulation  is  necessary  to  meet  the  iew  demands  of 
pilot  training,  certification,  and  currency  assurance  in  air  transportation. 
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PROLOGUE 


ltss  only  a  paper  moon 

Hanging  over  a  cardboard  sea. 

But  it  wouldn't  be  make  believe 
If  you'd  believe  in  me. 

It's  only  a  canvas  sky 

Sailing  over  a  muslin  tree. 

But  it  wouldn't  be  make  believe 
If  you'd  believe  in  me. 

Without  your  love. 

It's  a  honky  tonk  parade; 

Without  your  love. 

It's  a  melody  played  on  a  penny  arcade. 

It's  a  Bamum  and  Bailey  world. 

Just  as  phony  as  it  can  be. 

But  it  wouldn't  be  make  believe 
If  yau'd  believe  in  me. 

Once  popular  ballad  by  Harold  Arlen ,  Billy 
Rose }  and  Yip  Harburg 


Williges,  Roscoe,  and  Williges 


2 


BACKGROUND 

Flight  training  at  the  close  of  World  War  I  was  a  haphazard  process  at  best. 
Basically,  a  new  pilot  was  trained  solely  by  demonstration  and  exhortation 
improvised  by  his  particular  flight  instructor.  The  few  flight-trainer  type  devices 
in  use  were  short-winged  aircraft,  Incapable  of  flight,  known  as  "stub-winged 
Jennies"  or  "grass-cutters."  Students  would  run  these  early  trainers  up  and  down 
a  large  field  in  an  attempt  to  leam  to  control  the  craft. 

Since  those  early  days  of  flying,  ground-based  flight  simulators  and  trainers 
have  evolved  from  the  famous  Link  “Blue  Boxes"  of  World  War  II  into  precisely 
engineered  devices  capable  of  accurately  computing  the  aerodynamic  responses  of 
an  airplane  to  control  inputs  and  of  reproducing  realistic  cockpit  instrument  indica¬ 
tions  for  all  flight  situations.  But,  despite  the  sophistication  of  contemporary 
simulators  and  ths  longevity  of  their  use,  many  research  issues  concerning  ground- 
based  flight  simulators  and  trainers  remain  unanswered.  This  paper  is  a  review  of 
trainer-related  research  with  an  emphasis  on  these  unresolved  questions. 

Why  Simulate? 

Obviously,  the  initial  question  is  why  use  a  simulator  or  trainer  at  all. 
According  to  Gagne  (1962),  the  major  difference  between  a  simulator  and  the 
operational  situation  is  that  the  simulator  provides  its  users  with  greater  control 
over  ambient  conditions.  Whereas  the  real  world  is  subject  to  unpredictable 
variations,  a  simulator  provides  planned  variation  of  various  elements  of  the  real 
situation  with  unessential  variables  in  the  real  situation  omitted.  The  essential 
condition  for  effective  training  is  that  the  simulator  be  procedurally  faithful  to  the 
aircraft  it  is  designed  to  represent.  Determining  which  aspects  of  the  operational 
situation  can  reasonably  be  left  out  is  a  central  aspect  of  the  simulator  design 
process. 

The  second  major  advantage  offered  by  simulators  is  fhat  dangerous 
elements  in  the  operational  situation  may  be  represented  safely.  For  example, 
an  aircraft  simulator  might  represent  an  engine  on  fire  by  a  flashing  red  light 
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rather  than  by  an  actual  flame-producing  fire.  Furthermore,  emergency  procedures 
that  would  be  too  dangerous  to  teach  in  the  air  may  be  taught  safely  in  ground- 
based  devices. 

A  third  advantage  of  simulators  is  their  lower  operating  cost  in  comparison 
with  the  cost  of  operating  counterpart  aircraft.  Simulator  use  is  independent  of 
weather  or  time  of  day,  and  the  performance  of  individual  flight  tasks  and  procedures 
can  be  interrupted  and  repeated,  thereby  allowing  errors  to  be  corrected  immediately 
and  the  distribution  of  practice  on  sequentially  dependent  flight  tasks  to  be  optimized. 
For  these  reasons,  a  major  portion  of  any  flight  curriculum  can  be  taught  at  a  fraction 
of  the  cost  of  training  in  the  equivalent  aircraft. 

How  Can  Simulation  Be  Used  ? 

Flight  simulators  and  trainers  have  several  uses.  Initially,  performance 
in  a  simulator  or  trainer  can  be  used  in  pilot  selection  as  a  predictor  of  future 
success  in  training  and  operations.  Second,  a  simulator  that  reproduces  the  aerodynamic 
responses  of  an  aircraft  with  good  fidelity  is  valuable  for  teaching  new  psychomotor 
skills  required  for  operating  an  aircraft.  Furthermore,  the  training  functions  of 
simulators  are  not  limited  solely  to  initial  acquisition  of  flying  skills.  Trainers  can 
be  used  effectively  to  familiarize  an  experienced  pilot  with  the  operating  procedures 
and  characteristics  of  an  aircraft  to  which  he  is  newly  assigned.  Such  training  should 
reduce  the  transitioning  pilot's  initial  erroneous  responses  in  the  air  resulting  from 
negative  transfer  associated  with  the  need  to  make  different  responses  to  highly 
similar  stimuli.  In  addition,  simulators  can  be  used  both  to  reassess  and  to  maintain 
the  proficiency  of  licensed  pilots. 

Proficiency  assessment  in  a  flight  simulator  is  more  economical  and  more 
readily  controlled  than  a  similar  evaluation  in  the  air.  in  fact,  these  devices 
have  proven  to  be  so  useful  that  virtually  all  check  rides  for  airline  pilots  are 
given  in  simulators  rather  than  in  their  counterpart  aircraft.  Recently,  commercial 
airline  companies  have  conducted  research  to  determine  the  feasibility  of  increasing 
the  percentage  of  training  in  simulators  for  pilots  of  large  jet  aircraft.  Results  of 
studies  conducted  by  both  Trans  World  Airlines  (1969)  and  American  Airlines  (1969) 
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indicate  that  experienced  pilots  can  be  trained  to  type-rating  proficiency  entirely 
in  a  flight  simulator.  In  addition,  performance  evaluations  in  the  simulator 
accurately  predicted  performance  in  the  corresponding  aircraft.  These  studies 
suggest  that  the  Federal  Aviation  Administration  could  modify  pilot  certification 
requirements  to  allow  increased  use  of  simulation  equipment  of  proven  effectiveness. 

ISSUES  IN  FLIGHT  SIMULATOR  DESIGN 

Smode,  Hall,  and  Meyer  (1966)  have  compiled  a  relatively  comprehensive 
review  of  research  studies  using  flight  simulators  and  trainers  for  pilot  training  and 
have  indicated  areas  in  which  additional  research  is  needed.  Although  the  terms 
simulator  and  trainer  are  sometimes  used  interchangeably,  a  distinction  should  be 
made:  a  simulator  is  designed  to  represent  a  specific  counterpart  vehicle  or 
operational  situation;  a  trainer  is  intended  to  represent  a  class  of  vehicles  in 
various  situations.  Much  of  the  research  literature  on  simulation  in  pilot  training 
can  be  subdivided  into  two  areas:  degree  of  simulation  and  fidelity  of  simulation. 
Degree  of  simulation  refers  to  the  inclusion  of  design  features  such  as  motion, 
extracockpit  visual  cues,  and  part-task  versus  whole-task  representation.  Fidelity 
of  simulation  refers  to  the  accuracy  with  which  design  features  represent  or  dupli¬ 
cate  their  real-world  counterparts.  The  usual  reason  for  striving  for  high  fidelity  of 
simulation  is  to  maximize  transfer  of  training  to  performance  in  the  operational 
situation  (Muckier,  Nygaard,  O'Kelly,  and  Williams,  1959).  The  following  draws 
upon  Smode's  review  of  degree  of  simulation  and  Muckler's  analysis  of  fidelity  of 
simulation,  supplemented  by  ideas  of  our  own  concerning  issues  in  flight  simulation 
research. 

Degree  of  Simulation 

Motion.  Research  findings  dealing  with  motion  simulation  are  as  yet 
inconclusive.  Results  may  be  divided  into  three  categories:  those  that  support  the 
value  of  motion,  those  that  suggest  that  the  value  of  motion  depends  upon  the 
transfer  task,  and  those  that  suggest  that  the  value  of  motion  is  merely  a  transient 
effect. 
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Because  motion  provides  the  trainee  with  additional  cues,  many  researchers 
have  concluded  that  motion  facilitates  transfer  performance  (Besco,  1961;  Buckhout, 
Sherman,  Goldsmith,  and  Vitale,  1963;  Ruocco,  Vitale,  and  Binfari,  1965a; 

1965b;  Townsend,  1956).  However,  these  studies  evaluated  learning  with  criterion 
trials  in  a  simulator  rather  than  in  the  air. 

There  has  been  some  speculation  that  motion  is  strictly  necessary  only  in 
specific  situations,  where  acceleration  cues  either  improve  performance  by 
facilitating  anticipatory  responses  or  hinder  performance  by  making  it  more 
difficult  for  hie  pilot  to  make  necessary  control  adjustments  (Rathert,  Creer,  and 
Douvillier,  1959;  Rathert,  Creer,  and  Sadoff,  1961).  A  body  of  evidence  suggests 
that  although  motion  cues  do  seem  to  facilitate  an  initially  higher  level  of  perfor¬ 
mance,  this  effect  rapidly  fades  with  subsequent  flight  experience  (Caro  and 
Isley,  1966;  Feddersen,  1961). 

Several  studies  have  linked  the  value  of  motion  with  the  experience  level 
of  the  pilot.  Flexman  (1966)  and  Briggs  and  Wiener  (1959)  noted  that  because 
experienced  pilots  often  rely  on  motion  rather  than  instrument  readings,  motion 
becomes  more  important  as  experience  level  increases.  On  the  other  hand. 

Muckier,  Nygaard,  O'Kelly,  and  Williams  (1959)  suggested  that  motion  combined 
with  contact  cues  is  more  important  during  the  initial  stages  of  learning.  When 
visual  and  vestibular  cues  are  conflicting,  pilots  tend  to  rely  more  upon  their 
vestibular  cues  as  their  confidence  in  the  visual  information  decreases  (Johnson  and 
Williams,  1971). 

Smode,  Hall,  and  Meyer  (1966)  indicated  a  need  for  additional  research 
concerning  simulator  motion,  and  such  research  is  still  needed  today.  In  fact,  the 
question  of  whether  or  not  motion  cues  influence  transfer  at  all  is  as  yet  unanswered. 
Because  of  the  possibility  that  erroneous  motion  cues  might  actually  cause  negative 
transfer,  the  issue  of  whether  or  not  simulator  motion  is  beneficial  cannot  be 
separated  from  a  consideration  of  the  fidelity  of  motion  cues  necessary  to  produce 
positive  transfer  and  the  relative  transfer  effectiveness  and  the  cost-effectiveness 
tradeoff  of  increasing  motion-cue  fidelity.  Studies  are  also  needed  to  determine 
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the  relationship  between  cockpit  workload  and  the  value  of  motion  cues  of  varying 
experience  levels. 

Extracockpit  visual  cues.  Because  only  selected  visual  cues  are  used  by 
pilots,  and  visual  cues  are  not  required  to  perform  every  flying  maneuver,  the 
complete  external  visual  environment  does  not  need  to  be  reproduced  in  a  flight 
simulator.  The  real  problem  is  determining  exactly  which  visual  cues  are  necessary. 

An  early  study  at  the  University  of  Illinois  using  a  crude  contact  training 
device  confirmed  the  value  of  visual  cues  for  training  private  pilots  (Flexman, 
Matheny,  and  Brown,  1950).  Interestingly,  the  same  device  was  ineffective  for 
training  military  pilots  (Ornstein,  Nichols,  and  Flexman,  1954).  The  explanation 
given  for  this  difference  was  that  the  value  of  extracockpit  cues  is  limited  by  the 
quality  of  the  instruction  associated  with  their  use. 

Studies  using  slightly  more  complex  extracockpit  visual  devices  have  con¬ 
firmed  their  value  (Creelman,  1955;  Payne,  Dougherty,  Hosier,  Skeen,  Brown, 
and  Williams,  1954).  However,  the  value  of  extracockpit  visual  simulation  in  the 
learning  of  perceptual  responses  in  flying  in  the  absence  of  related  psychomotor 
responses  has  not  been  substantiated  (Creelman,  1955;  Adams  and  Hufford,  1961). 

A  second  group  of  studies  has  been  concerned  with  training  pilots  to  divide 
their  attention  between  external  visual  cues  and  the  instruments  within  the  aircraft. 
Pfeiffer,  Clark,  and  Danaher  (1963)  concluded  that  training  does  improve  a  pilot's 
time-sharing  ability.  Further,  such  training  can  be  given  in  relatively  inexpensive 
training  devices  (Gabriel,  Burrows,  and  Abbott,  1965).  The  questions  of  whether 
or  not  the  time-sharing  skills  learned  in  such  devices  transfer  to  flight  and  are 
retained  ovw  extended  periods  remain  to  be  answered. 

Smode,  Hall,  and  Meyer  (1966)  point  out  that  information  on  the  value  of 
contact  devices  is  muddled  because  the  utility  of  the  visual  device  is  so  intertangled 
with  the  fidelity  of  the  particular  simulator  being  used.  They  indicate  that  more 
information  is  needed  to  clarify  the  value  of  contact  displays  both  as  a  part  of  and 
independent  of  the  specific  simulator  used.  In  addition,  satisfactory  methods  of 
presenting  v!"ual  and  motion  cues  simultaneously  need  to  be  explored.  For 
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example,  is  moving  the  visual  display  in  response  to  the  pilot's  input  equivalent 
perceptually  tc  moving  the  trainer?  A  related  issue  is  the  value  of  open-loop 
training  for  closed-loop  tasks.  Again,  the  factor  of  level  of  pilot  experience 
needs  to  be  investigated.  Finally,  the  effectiveness  of  inexpensive  devices  to 
teach  time-sharing  should  be  further  explored,  and  the  value  of  tachistoscopic 
training  to  increase  instrument  reading  speeds  needs  to  be  determined. 

Part-task  versus  whole-task  training.  Although  a  great  deal  of  research  has 
been  concerned  with  part-task  versus  whole-task  learning,  the  results  of  this 
research  have  limited  applicability  to  pilot  training.  Traditionally,  part-task 
training  in  verbal  and  simple  motor  tasks  involves  the  development  of  component 
parts  of  a  skill  and  subsequent  practice  on  all  parts  concurrently.  Part-task  pilot 
training  is  typically  molecular  rather  than  atomic,  often  involving  training  on 
individual  whole  tasks  that  are  later  practiced  in  series  with  different  tasks  rather 
than  in  parallel  with  other  parts  of  the  same  task.  An  example  of  a  part-task  flight 
trainer  is  a  device  that  simulates  with  high  fidelity  only  the  attack  phase  of  an 
air-to-air  intercept  mission  (Nygaard  and  Roscoe,  1953). 

The  results  of  several  studies  specifically  concerned  with  part-task  trainers 
for  pilots  have  supported  the  utility  of  such  devices  (Dougherty,  Houston,  and 
Nicklas,  1957;  Miller,  i960;  Parker  and  Downs,  1961;  Pomarolli,  1965).  A 
frequent  limitation  of  part-task  trainers  is  that  they  fail  to  provide  an  opportunity 
for  practice  in  time-sharing  attention  among  tasks  (Adams,  Hufford,  and  Dunlop, 
i960;  Hufford  and  Adams,  1961).  A  subsequent  period  of  integration  is  necessary 
to  allow  students  to  perform  various  subtasks  on  a  time-shared  basis.  On  the 
positive  side,  part-task  training  seems  to  require  less  relearning  after  a  period  of 
rest  (Hufford  and  Adams,  1961). 

Smode,  Hall,  and  Meyer  (1966)  suggest  two  areas  for  future  research. 

First,  the  relative  contribution;  of  part-task  and  whole-task  trainers  need  to  be 
determined  so  that  less  expensive  trainers  can  be  used  whenever  appropriate.  In 
addition,  information  concerning  how  task  integration  proceeds  is  needed. 
Eventually  a  pilot  must  scan  his  instruments,  tune  radio  receivers,  navigate,  and 
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communicate,  all  while  flying  his  airplane.  Occasionally  he  may  have  to  disarm 
a  hijacker.  Practice  in  doing  all  such  things  concurrently  is  required  in  a  com¬ 
prehensive  training  program. 

Because  almost  any  training  must  be  part-task  training  to  some  degree,  the 
real  issue  would  appear  to  be  the  optimum  size  for  each  learning  "chunk." 
Obviously,  very  few  people  get  into  a  helicopter  for  the  first  time  and  solo;  rather, 
training  proceeds  in  steps.  Optimum  step  size  in  pilot  training  is  an  open  question. 

Fidelity  of  Simulation 

Muckier,  Nygaard,  O' Kelly,  and  Williams  (1959)  made  a  thorough  review 
of  early  findings  concerning  the  fidelity  of  simulation  necessary  for  maximum 
transfer  and  found  widely  varying  results.  Several  studies  led  to  the  conclusion 
that  fidelity  of  simulation  made  little  difference  in  the  amount  of  transfer  to  the 
air.  Mahler  and  Bennett  (1950)  found  no  differences  in  transfer  among  several 
training  devices  varying  widely  in  fidelity.  With  the  exception  of  performance  on 
one  maneuver,  recovery  from  unusual  attitudes,  Wilcoxon,  Davy,  and  Webster's 
(1954)  results  support  Mahler  and  Bennett. 

On  the  other  hand,  a  study  by  Omstein,  Nichols,  and  Flexman  (1954) 
isolated  particular  components  of  the  pilot's  task  and  found  that  training  in  a  simu¬ 
lator  of  higher  fidelity  (Link  P-1)  consistently  resulted  in  better  transfer  on  each  of 
22  instrument  maneuvers  than  training  in  either  the  Link  AN-T-18  or  C-8  trainers. 
Similarly,  the  results  of  Dougherty,  Houston,  and  Nicklas  (1957)  favor  trainers  of 
higher  fidelity.  They  found  better  transfer  to  the  SNJ  aircraft  when  pilots  were 
trained  either  in  an  SNJ  operational  flight  trainer  or  a  procedural  trainer  than  with 
a  photographic  mockup.  However,  in  this  study  the  advantage  enjoyed  initially  by 
trainers  of  higher  fidelity  was  negligible  by  the  sixth  air  trial. 

Each  of  the  preceding  four  studies  was  conducted  under  similar  experimental 
conditions,  but  the  results  are  irreconcilable.  Muckier,  Nygaard,  O' Kelly,  and 
Williams  (1959)  concluded  that  studies  concerned  with  fidelity  of  simulation  are 
plagued  by  a  variety  of  problems  such  as  lack  of  general izability  from  oversimpii 
fied  laboratory  tasks  and  inadequate  measurement  techniques.  Before  any  definite 
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conclusions  can  be  drawn  about  fidelity  of  simulation,  more  detailed  information  is 
needed  to  determine  how  such  variables  as  instructor  ability,  variations  in  the 
difficulty  of  the  training  task,  and  pilot  experience  level  affect  transfer  perfor¬ 
mance.  As  stated  previously,  there  has  been  no  experimental  effort  to  determine 
the  relationship  between  transfer  of  training  and  fidelity  of  simulated  motion  cues. 

ISSUES  IN  FLIGHT  SIMULATOR  EVALUATION 

What  to  Measure  and  How? 

As  noted  by  Blaiwes  and  Regan  (1970)  and  more  recently  by  Roscoe  (in 
McGrath  and  Harris,  1971),  three  criteria  must  be  considered  in  properly  evalu¬ 
ating  any  training  device:  (1)  efficiency  of  original  learning,  (2)  transfer  of  what 
was  learned  in  one  situation  to  another,  and  (3)  retention  of  what  was  once  learned. 

Original  learning.  To  determine  the  effectiveness  of  synthetic  flight 
trainers  against  any  of  these  criteria,  objective  performance  measures  are 
necessary.  One  traditional  measure  of  learning  is  instructor  ratings.  In  general, 
such  ratings  tend  to  be  subjective  and  as  such  are  hampered  by  gross  inconsistencies 
among  independent  observers.  In  an  attempt  to  overcome  many  of  the  difficulties 
associated  with  subjective  grading  by  check  pilots,  the  development  of  objective 
flight  inventories  has  been  encouraged.  One  of  the  first  of  these  was  the  Ohio 
State  Flight  Inventory  which  combined  a  series  of  five-point  rating  scales  for  each 
maneuver  with  some  objectively  scored  items  completed  during  flight.  Ericksen 
(1952)  summarized  studies  using  this  inventory. 

In  1947  an  extensive  program  to  develop  an  objective  checklist  for  pilot 
evaluation  was  begun  under  the  sponsorship  of  the  CAA  through  the  National 
Research  Council  Committee  on  Aviation  Psychology  (Gordon,  1947;  1949).  The 
decisions  to  include  items  were  based  upon  critical  incidents,  accident  reports, 
and  job  analyses.  Tasks  evaluated  were  arranged  into  a  standard  flight  sequence, 
including  both  subjective  and  objective  items.  To  maximize  objectivity,  graphics 
or  pictures,  quantitative  data,  and  precise  descriptions  were  used. 
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Another  flight  inventory,  developed  at  the  Human  Resources  Research 
Office,  had  as  a  goal  the  complete  description  of  pilot  performance  (Smith, 
Flexman,  and  Houston,  1952).  Based  on  reported  critical  behaviors  in  flying,  the 
HUMRRO  entory  consisted  of  two  types  of  items:  scale  items  (whether  or  not 
within  predetermined  tolerances)  and  categorical  items  .whether  or  nor  completed). 
Although  its  use  has  been  limited  to  research,  the  inventory  has  provided  reliable 
normative  data  to  set  standards  for  pilot  training. 

More  recently,  the  Illinois  P r  <ate  Pilot  Performance  Scale  has  been 
developed  (Povenmire,  Alvares,  and  Jamos,  1970;  Po/enmire  and  Roscoe,  1973; 
Selzer,  Hulin,  Alvares,  Swartzendruber,  and  Roscoe,  1972).  This  scale  evaluates 
performance  on  each  of  ten  maneuvers  from  the  FAA's  Private  Pilot  flight  test 
guide.  Four  to  six  quantitative  variables  for  each  maneuver  are  scored  by  marking 
the  maximum  deviation  from  desired  performance  on  appropriate  scales.  Equal 
weighting  is  given  to  all  variables  measured.  Individual  deviation  scores  are 
converted  into  standard  score,  based  upon  the  observed  variability  among  students 
tested  for  Private  Pilot  certification  at  the  University  of  Illinois.  Observer- 
observer  reliability  in  excess  of  .80  has  been  found  for  this  testing  instrument 
(Selzer,  Hulin,  Alvares,  Swartzendruber,  and  Roicoe,  1972). 

Despite  the  increasing  objectivity  of  pilot  performance  grading,  the 
reliability  of  the  so-called  objective  checks  has  been  disappointing  in  routine  use. 
According  to  Smode,  Hall,  and  Meyer  (1966),  ieveral  factors  contribute  to  the 
limited  capability  of  objective  measures.  They  are:  check  pilot  biases,  inadequate 
descriptions  of  acceptable  performance,  low  validity  based  upon  the  failure  to 
define  precisely  the  critical  skills  to  be  assessed,  and  the  need  to  give  special 
training  to  check  pilots  on  the  use  of  objective  measurements. 

The  shortcomings  of  check  pilot  ratings  have  given  impetus  to  the 
development  of  automatic  recording  devices  built  into  synthetic  trainers. 
Danneskiold  (1955)  conducted  a  study  to  determine  the  feasibility  of  mechanical 
scoring  devices.  Although  accuracy  of  measurement  was  an  asset,  the  mechanical 
devices  were  limited  by  inflexibility,  cumbersome  size,  and  the  failure  to  reflect 
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meaningful  aspects  of  flight.  Current  research  on  automated  performance  measure¬ 
ment  by  Knoop  (in  McGrath  and  Harris,  1971)  hos  thus  far  resulted  only  in  tenta¬ 
tive  conclusions  about  requirements  and  feasibility.  However,  it  is  evident  that 
semiautomatic  performance  assessment  methods  resulting  from  this  research  will 
receive  considerable  attention  in  connection  with  the  evaluation  of  the  effective¬ 
ness  of  new  synthetic  training  devices. 

The  most  elusive  problem  in  the  semiautomatic  assessment  of  pilot  perfor¬ 
mance  is  determining  what  ideal  pilot  behavior  is.  At  present  the  real-world 
criterion  most  often  used  seems  to  be  expert  judgments  of  what  maneuvers  are 
essential  and  what  range  of  performance  variation  can  be  tolerated.  Another 
approach  is  the  collection  of  normative  data  from  the  performance  of  experienced 
pilots.  However,  FI  exman  asserts  that  the  variance  among  experienced  pilots  is 
greater  than  that  among  student  pilots  (in  McGrath  and  Harris,  1971).  Until  some 
agreement  is  reached  about  what  constitutes  ideal  pilot  performance,  evaluative 
techniques  that  measure  deviations  from  a  standard  will  be  severely  limited. 

Transfer.  A  critical  measure  of  the  effectiveness  of  flight  simulators  is 
the! i'  transfer  to  performance  in  the  air.  Although  early  evaluations  of  flight 
trainers  provided  estimates  of  air  time  savings,  many  failed  to  include  a  control 
group,  eliminating  any  objective  measure  of  transfer  of  training  (Conlon,  1939; 
Crannell,  Greene,  and  Chamberlain,  1941;  Greene,  1941). 

A  number  of  studies  conducted  at  the  University  of  Illinois  were  designed 
to  measure  the  value  of  synthetic  training  in  reducing  the  flight  hours  necessary  to 
obtain  a  ftivate  Pilot’s  license.  Williams  and  Flexman  (1949)  measured  the  amount 
of  flight  time  until  students  were  judged  ready  to  "solo."  The  results  revealed  no 
significant  differences  among  groups  of  subjects  having  zero,  two,  or  four  hours  cr 
experience  in  a  C-3  Link  trainer.  The  experimenters  recognized  that  the  amount 
of  flight  time  until  ready  to  solo  was  not  a  good  criterion  for  the  evaluation  of  an 
early  Link  trainer,  because  skills  required  in  landing  and  other  oresolo  maneuvers 
requiring  visual  cues  other  than  a  horizon  line  were  not  easily  taught  in  this  type 
of  trainer. 


Williges,  Roscoe,  and  Williges 


12 


in  their  second  study,  evaluating  the  School  Link,  Williams  ond  Flexman 
(1949)  used  three  errorless  trials  on  selected  maneuvers  as  the  criterion  measure. 
The  experimental  group  that  received  both  Link  training  and  inflight  training 
learned  the  maneuvers  to  criterion  with  28  percent  fewer  air  trials  and  22  percent 
fewer  errors  in  flight  than  a  control  group  receiving  only  inflight  training.  The 
experimenters  suggested  that  approximately  25  percent  of  beginning  flight  training 
could  be  accomplished  on  the  ground. 

A  further  study  by  Flexman,  Matheny,  and  Brown  (1950)  compared  two 
groups  of  student  pilots  after  ten  hours  of  flight  training.  One  group  received  no 
Link  training  and  a  second  group  received  whatever  Link  training  each  individual 
student  considered  to  be  beneficial.  Results  indicated  that  the  Link  group  was 
more  proficient  on  a  flight  examination  similar  to  the  Private  Pilot  Performance 
Scale. 

At  about  the  same  time  that  Link  trainers  were  undergoing  these  evalua¬ 
tions,  similar  studies  were  conducted  with  the  Link  P-1  (SNJ^  simulator  which 
approximated  a  military  aircraft,  the  T-6  (SNJ).  In  general,  students  receiving 
partial  synthetic  training  performed  as  well  or  better  than  students  trained  solely 
in  the  air.  Comparisons  were  based  upon  various  criteria  including  number  of 
flight  failures  and  accidents,  check  flight  grades,  and  total  training  hours 
(Flexman,  Townsend,  and  Ornstein,  1954;  Mahler  and  Bennett,  1950;  Ornstein, 
Nichols,  and  Flexman,  1954;  and  Wilcoxon,  Davy,  and  Webster,  1954). 

The  first  and  only  studies  that  have  allowed  an  assessment  of  transfer 
effectiveness  of  a  specific  flight  simulator  to  its  counterpart  airplane  on  a 
maneuver-by-maneuver  basis  were  conducted  in  1950  by  Flexman  and  reported  22 
years  later  (Flexman,  Roscoe,  Williams,  and  Williges,  1972). 

In  1969,  almost  20  years  later,  Povenmire  and  Roscoe  (1971)  measured  the 
transfer  effectiveness  of  the  relatively  new  Link  GAT-1  and  the  Link  AN-T-18  of 
World  War  II  as  used  by  typical  flight  instructors  in  a  routine  private  pilot  training 
program.  Eleven  hours  of  ground  training  in  the  GAT-1  saved  an  average  of  1 1 
hours  (34.5  versus  45.5  for  the  control  group)  in  the  Piper  Cherokee,  thereby 
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yielding  a  Transfer  Effectiveness  Ratio  of  1.00.  An  equnl  amount  of  training  in 
the  famous  and  venerable  AN-T-18  saved  an  average  of  nine  hours  in  the  Cherokee 
for  a  Transfer  Effectiveness  Ratio  of  0.82,  thereby  providing  further  justification 
for  its  continued  widespread  use  more  than  30  years  after  its  invention. 

Retention.  Interestingly,  the  most  common  measure  of  training  effective¬ 
ness,  retention  of  material  learned,  has  been  generally  ignored  in  the  evaluation  of 
simulators.  Most  studies  fail  to  measure  the  permanence  of  simulator  learning, 
despite  the  obvious  importance  of  retaining  flying  skills.  One  notable  exception  is 
a  study  by  Mengelkoch,  Adams,  and  Gainer  (19581  which  measured  simulator  per¬ 
formance  after  a  four-month  retention  interval.  Unfortunately,  both  training  and 
retention  trials  were  conducted  solely  in  a  trainer  with  no  measure  of  performance 
in  the  air. 

Other  studies  in  pilot  training  have  not  been  designed  to  use  the  retention 
scores  obtained  as  a  measure  of  the  effectiveness  of  various  types  and  amounts  of 
original  learning  including  simulator  training  (Seltzer,  1970).  Measurement  of 
retention  is  hindered  by  such  problems  as  variations  in  the  original  training  of 
subjects,  difficulty  of  controlling  the  amount  of  flying  experience  each  individual 
pilot  receives  during  the  retention  period,  and  unavailability  of  subjects  after  a 
sufficiently  long  retention  period.  The  lack  of  simulator  studies  using  a  retention 
measure  reflects  the  general  insufficiency  of  information  relating  to  retention  of 
pilot  skills  or,  for  that  matter,  retention  of  any  complex  motor  skill. 

Cost  Effectiveness 

A  trend  in  simulator  development  has  been  to  duplicate  as  closely  as 
possible  every  detail  of  the  operational  aircraft.  As  hardware  technology  develops, 
new  capabilities  are  added  to  flight  simulators  resulting  in  a  rapid  cost  spiral. 
Unfortunately,  the  training  value  of  each  added  capability  is  seldom  assessed. 

With  inflated  equipment  costs,  the  need  to  weigh  the  relative  value  of  physical 
fidelity  against  its  cost  has  become  evident. 

In  most  of  the  quantitative  transfer  studies  with  simulators,  the  speed  of 
learning  by  an  experimental  group,  previously  trained  to  a  specified  level  of 
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proficiency  in  a  simulator,  has  been  compared  with  that  of  a  control  group 
receiving  no  simulator  training.  Transfer  has  been  measured  solely  by  the  saving 
in  flight  time  or  reduction  in  errors  with  no  regard  for  the  amount  of  simulator 
training  given  members  of  the  experimental  group. 

The  study  by  Povenmire  and  Roscoe  (1971),  in  which  subjects  were  given  a 
fixed  amount  of  ground  training,  was  an  exception.  It  is  doubtful  that  anyone 
would  seriously  propose  replacing  the  GAT-1  with  the  AN-T-18  in  a  modern 
Private  Pilot  training  program;  nevertheless,  a  strong  case  could  be  made  for  doing 
so,  as  shown  below  in  a  cost  effectiveness  analysis  based  on  the  Povenmire  and 
Roscoe  data. 

Assume  the  hourly  cost  of  dual  flight  training  to  be  $22  ($14  for  the 
Cherokee  plus  $8  for  the  instructor)  and  the  corresponding  values  for  the  GAT-1 
and  the  AN-T-18  to  be  $16  ($8  +  $8)  and  $10  ($2  +  $8),  respectively.  In  a  flight 
course  normally  requiring  46  hours  in  the  air,  if  11  hours  of  training  in  the  GAT-1, 
costing  $176,  save  1 1  hours  in  the  Cherokee,  costing  $242,  each  $1.00  spent  in 
the  GAT-1  buys  $1.38  worth  of  air  training.  Similarly,  if  11  hours  of  training  in 
the  AN-T-18  costing  $110,  save  9  hours  in  the  Cherokee,  costing  $198,  each 
$1.00  spent  in  the  AN-T-18  saves  $1.80  in  the  air. 

Determining  Essential  Realism 

Several  approaches  to  lowering  equipment  costs  are  possible.  The  first 
requires  a  realistic  appraisal  of  the  amount  of  reaKsm  essential  for  the  training 
task.  Too  often  factors  adding  realism  to  a  simulator  are  evaluated  strictly  in  a 
go/no-go  fashion.  For  example,  the  research  question  generally  has  been  whether 
or  not  to  include  extracockpit  visual  displays,  rather  than  what  visual  cues  are 
necessary  to  achieve  high  transfer  to  flight. 

Payne,  Dougherty,  Hosier,  Skeen,  Brown,  and  Williams  (1954)  used  a 
relatively  simple  visual  display,  providing  only  a  dynamic  perspective  outline  of  a 
runway  on  a  screen  in  front  of  a  l-CA-2  (SNJ)  simulator,  to  prepare  a  group  of 
beginning  students  for  solo  flight  in  a  T-6  aircraft.  The  transfer  group  reached 
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proficiency  in  landing  with  61  percent  fewer  air  trials  and  74  percent  fewer  errors 
in  landing  approaches  than  did  a  control  group  trained  only  in  the  aircraft. 

The  inclusion  of  motion  in  present-day  simulators  is  another  example  of  the 
realism-versus-cost  problem.  Cohen  (1970)  estimated  the  cost  of  a  three-degree- 
of-freedom  motion  system  at  about  $100,000  and  a  si x-degree-of- freedom  system 
at  $250,000.  Such  costs  are  not  insignificant;  nevertheless,  most  large-aircraft 
simulators  have  complex  motion  systems  even  though  there  is  little  evidence  to 
indicate  that  such  motion  capability  significantly  improves  ground-based  training, 
and  much  of  a  pilot's  training  encourages  him  to  disregard  acceleration  cues  in 
flight. 

Cohen  (1970)  indicated  that  a  systematic  research  effort  is  needed  to 
determine  what  kinds  and  what  degrees  of  motion  are  essential  for  the  flight 
training  iask.  An  initial  effort  in  this  regard  might  be  to  determine  what  aspects 
of  motion  a  pilot  can  perceive  and  how  acceleration  thresholds  vary  under  stress. 
Obviously,  if  certain  types  of  motion  cues  cannot  be  perceived  by  the  human 
operator,  providing  them  is  at  best  wasteful.  In  addition,  if  motion  of  some  sort  is 
included  in  a  simulator,  an  effort  should  be  made  to  avoid  introducing  misleading 
cues  that  hinder  rather  than  facilitate  transfer. 

In  view  of  the  large  sums  invested  in  the  design,  development,  and  pro¬ 
duction  of  complex  simulator  motion  systems,  it  is  difficult  to  understand  why  there 
has  been  no  objective,  controlled  experiment  to  assess  their  transfer  effectiveness. 
An  experiment  by  Matheny,  Dougherty,  and  Willis  (1963)  showed  that  relatively 
faithful  cockpit  motion  improves  pilot  performance  in  the  simulator,  presumably 
by  providing  alerting  cues,  and  recent  experiments  at  Ames  Research  Center 
(Guercio  and  Wall,  1972)  and  at  the  Aviation  Research  Laboratory  of  the  University 
of  Illinois  (Roscoe,  Denney,  and  Johnson,  1971;  Jacobs,  Williges,  and  Roscoe, 
1972)  support  this  finding.  However,  there  is  no  evidence  one  way  or  the  other  to 
indicate  that  this  improvement  transfers  to  flight.  The  general  experimental  finding 
that  relatively  difficult  training  tasks  yield  higher  transfer  than  easier  ones  suggests 
that  transfer  might  be  reduced  as  a  consequence  of  adding  motion  cues  that  make 
the  simulated  flight  task  easier. 
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The  evident  reason  that  large  sums  are  spent  for  simulator  motion  systems, 
with  no  evidence  of  their  training  value,  is  their  high  face  validity.  A  high- 
fidelity  motion  system  is  a  delight  to  any  pilot;  the  illusion  of  flight  is  extremely 
realistic.  The  decision  to  include  a  complex  motion  system  in  a  simulator  is 
invariably  determined  by  the  enthusiasm  of  pilots,  particularly  ones  in  high  places. 

Training  Objectives  and  Low-Cost  Trainers 

A  related  approach  to  lowering  equipment  costs  involves  identifying  aspects 
of  flight  training  that  can  be  taught  in  low-cost  devices.  When  a  less  complex 
training  device  is  appropriate,  it  should  not  be  overlooked  in  favor  of  complex 
simulators  with  higher  face  validity. 

The  failure  to  consider  low-cost  devices  when  procuring  flight  training 
equipment  was  well  illustrated  anecdotally  by  Prophet  (1966).  A  procedures 
trainer  that  had  cost  over  $100,000  was  pitted  against  a  plywood,  phonographic 
instrument-panel  mockup  costing  less  than  $100.  As  predicted  by  Prophet,  the 
static  mockup  fared  as  well  as  the  costlier  model  for  teaching  cockpit  procedures. 
Surprisingly,  the  mockup  trainees  also  did  as  well  as  the  simulator  trainees  on 
other  tasks  such  as  reading  instruments  and  making  precise  control  settings. 
Although  the  more  expensive  trainer  had  capabilities  far  beyond  the  scope  of  the 
mockup,  training  in  the  costlier  device  should  be  devoted  primarily  to  task 
elements  that  cannot  be  taught  effectively  with  less  costly  equipment. 

The  value  of  less  than  full  simulation  in  a  variety  of  flight  training 
situations  is  obvious;  cost  reductions  in  training  equipment  may  be  quite  large. 
However,  the  development  and  evaluation  of  simple  training  devices  depend  upon 
the  imagination  of  the  designer,  the  ingc-vity  of  the  instructor,  and  the  financial 
support  of  the  potential  user. 

INNOVATIONS  IN  SYNTHETIC  FLIGHT  TRAINING  AND  RESEARCH 

Although  the  unresolved  issues  in  synthetic  flight  training  all  have  their 
origins  in  relative  antiquity,  progress  toward  resolution  is  evident,  terms  such  as 
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adaptive  training,  computer-assisted  instruction,  cross-adaptive  measurement  of 
residua!  attention,  computer  gr<~!‘«~s,  incremental  transfer  effectiveness,  and 
response  surface  methodology  have  blossomed  during  the  past  decade.  Progress  in 
each  of  these  areas  shows  positive  acceleration. 

Adaptive  Training 

Although  all  personalized  instruction  is  in  a  sense  adaptive,  the  term 
adapt've  training  has  come  to  refer  specifically  to  the  automatic  adjustment  of  the 
difficulty,  complexity,  or  newness  of  a  training  task  as  a  function  of  the  individual 
student's  progress.  Automatically  increasing  the  average  amplitude  of  the  forcing 
function  for  a  tracking  task  as  a  student  learns,  requiring  a  student  to  handle  more 
and  more  subtasks  simultaneously  in  accordance  with  his  immediately  preceding 
performance,  and  introducing  new  and  different  tasks  as  old  tasks  are  mastered 
are  all  examples  of  adaptive  training. 

Adaptive  training  employs  predetermined  decision  rules  for  the  adjustment 
of  a  training  system  to  the  requirements  of  the  individual  trainee.  Subsequent 
system  outputs  are  determined  by  the  previous  output  from  the  student.  In  effect, 
task  difficulty  is  programmed  to  increase  appropriately  with  increasing  student 
proficiency. 

The  first  formal  application  of  automatically  adaptive  logic  to  the  training 
of  pilots  has  been  incorporated  into  the  Synthetic  Flight  Training  System  (SFTS) 
developed  by  the  Naval  Training  Device  Center  for  helicopter  pilot  training  by 
the  United  States  Army.  In  this  system,  one  central  digital  computer  drives  four 
cockpit  simulators  in  which  four  pilots  learn  to  fly  simultaneously  under  the 
supervision  of  a  single  instructor.  The  difficulty  of  certain  flight  tasks  adapts 
automatically  to  the  individual  student's  continuously  measured  performance. 

The  application  of  automatic  adaptation  of  task  difficulty  to  the  SFTS 
(Caro,  1969)  was  inspired  mainly  by  the  studies  of  Hudson  (1964)  and  Kelley 
(1966).  In  a  conference  on  adaptive  training  held  at  the  University  of  Illinois  in 
1970  (McGrath  and  Harris,  1971),  it  became  evident  that  a  central  issue  was  the 
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nature  of  the  adaptive  logic  employed.  Specifically,  should  error  limits  be  held 
constant  as  skill  increases  and  the  task  becomes  more  difficult,  as  advocated  by 
Kelley,  or  should  error  limits  vary  as  the  individual's  performance  improves,  as 
advocated  by  Hudson?  This  issue  is  currently  under  investigation  at  the  University 
of  Illinois  (Crooks,  1 97 1 ) . 

Computer- Assisted  Instruction 

Automated  adaptive  skill  training  is  a  form  of  computer-managed  instruction 
as  is  programmed  cognitive  training,  which  may  or  may  not  be  adaptive.  However, 
the  term  computer-assisted  instruction  (CAI)  implies  programmed  c  ,gnitive  learning 
in  which  an  automatically  branching  logic  allows  each  student  to  progress  through 
a  course  at  his  own  rate. 

The  application  of  CAI  to  the  ground-school  portion  of  the  flight  curriculum 
at  the  Institute  of  Aviation  of  the  University  of  Illinois  is  currently  in  progress. 
Courses  designed  to  prepare  students  for  Private,  Commercial,  Instrument,  Instructor, 
and  Airline  Transport  Pilot  certificates  and  ratings  will  be  programmed  for  the 
PLATO  system  which  eventually  will  have  terminals  throughout  the  nation. 

PLATO  is  the  acronym  for  Programmed  Logic  for  Automatic  Teaching  Operations. 

The  PLATO  system  (Bitzer  and  Johnson,  1971)  was  designed  to  aid  both 
student  and  instructor  in  the  educational  process  through  use  of  the  capabilities  of 
the  modern  digital  computer.  The  PLATO  computer  interacts  with  each  student  by 
presenting  information  and  reacting  to  student  responses.  The  actions  of  the  com¬ 
puter  follow  the  instructor's  rules  which  specify  what  is  to  be  done  in  each  and 
every  possible  situation.  A  lesson  constructed  of  such  a  set  of  rules  can  have  a 
flexibility  approaching  that  possible  when  each  student  has  a  human  tutor.  In 
fact,  the  rules  defining  a  useful  tutorial  lesson  presented  by  computer  are  quite 
similar  to  those  implicitly  used  by  a  human  teacher.  For  example,  areas  in  which 
a  student  has  proven  competence  are  given  minimal  coverage,  whereas  areas  in 
which  the  student  lacks  competence  are  developed  more  thoroughly. 

In  contrast  to  a  conventional  classroom  in  which  a  teacher  manages  20  to 
30  students  simultaneously  and  can  seldom  give  special  attention  to  individual 
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students,  PLATO  appears  to  give  each  student  undivided  attention.  This 
appearance  results  from  the  ability  of  the  computer  to  identify  and  handle  most 
student  requests  in  a  small  fraction  of  a  second.  When  several  students  request 
material  simultaneously,  the  PLATO  system  processes  their  requests  in  turn. 

However,  the  last  processed  student  seldom  has  to  wait  more  than  one-tenth  of  a 
second  for  a  reply  from  the  computer.  To  most  students,  one-tenth  of  a  second 
appears  to  be  instantaneous.  One  aspect  of  individual  attention  is  rapid  feedback. 
The  student  receives  immediate  knowledge  of  the  correctness  of  his  responses. 

The  primary  application  of  PLATO  to  the  training,  certification,  and 
currency  assurance  of  pilots  v.ii  be  in  the  cognitive  domain,  although  PLATO  is 
c/so  capable  of  certain  types  of  perceptual -motor  training.  The  individual  attention 
capability  of  PLATO  together  with  computational  and  graphic  display  abilities 
allow  authors  of  ground  school  courses  to  select  and  present  stored  material,  such 
as  special  characters,  photographic  slides,  and  either  printed  or  audio  messages, 
and  to  construct  geometric  figures  or  graphs  activated  by  instructions  of  either  the 
author  or  the  student.  A  constructed  graphic  display,  for  example,  might  be  used 
to  allow  a  itudent  in  an  aviation  course  to  specify  the  shape  and  construction  of 
an  airfoil.  PLATO  could  then  produce  a  cross-sectional  view  of  the  airfoil  on  the 
student's  plasma  display  screen.  Upon  request,  PLATO  might  also  show  the  paths 
of  air  molecules  flowing  around  the  airfoil  in  flight. 

As  the  number  of  terminals  grows  throughout  the  country,  it  will  become 
increasingly  possible  and  desirable  to  leave  much  of  the  certification  and  currency 
assurance  testing  to  the  PLATO  system.  Doing  this  would  allow  students  to  take 
FAA  tests  at  their  own  convenience  and  would  also  free  many  FAA  examiners  for 
more  important  tasks.  When  legislation  is  passed  requiring  all  pilots  to  undergo 
periodic  recertification,  the  extra  load  on  the  present  testing  system  is  going  to 
be  enormous.  Using  CAI  techniques  to  conduct  these  tests  will  provide  great  relief 
to  the  system. 
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Cross- Adaptive  Measurement  of  Residual  Attention 

The  automatically  adaptive  measurement  of  a  pilot's  "residual  attention" 
while  performing  routine  flight  tasks  can  consist  of  anything  from  rhythmic  tapping 
on  a  microswitch  with  a  finger  or  foot  (Mlchon,  1966)  to  complex  information 
processing  (Ekstrom,  1962;  Knowles,  1963).  Such  tasks  serve  at  least  two  functions. 
They  provide  an  inferential  measure  of  the  pilot's  mastery  of  the  primary  task,  and 
they  can  create  realistically  elevated  workload  pressures  typical  of  those  encountered 
in  flight  emergencies.  The  demands  imposed  by  such  tasks  can  be  made  to  cross- 
adapt  automatically  to  the  pilot's  performance  of  his  primary  flight  control  task. 

The  better  he  flys,  the  faster  flows  the  information  to  be  processed.  In  this  way 
the  pilot's  total  cockpit  workload  capacity  can  be  measured  as  a  function  of  his 
level  of  training  or  the  decay  in  his  proficiency  following  periods  of  inactivity. 

The  use  of  automatically  adaptive  and  cross-adaptive  secondary  tasks  for 
the  measurement  of  residual  attention  has  been  applied  both  in  the  experimental 
study  of  flight  display  and  control  design  variables  (Kraus  and  Roscoe,  1972)  and 
in  the  prediction  of  success  in  pilot  training  (Damos,  1972).  From  these  experi¬ 
ments,  it  has  become  evident  that  the  technique  also  can  produce  a  powerful 
instructional  effect  in  the  important  areas  of  attention  sharing  and  decision  making. 
Furthermore,  it  is  well  established  that  pilots  show  small  decrements  in  flying  skills 
over  long  periods  of  inactivity  but  show  large  decrements  quickly  in  procedural 
efficiency,  particularly  in  situations  requiring  attention  sharing  and  rapid  decision 
making. 

Thus,  residual  attention  tasks  provide  not  only  a  measure  of  the  initial 
attainment  of  proficiency  but  also  a  quick  and  reliable  means  of  testing  the  currency 
of  certificated  pilots.  Tasks  similar  to  those  already  employed  effectively  in 
human  engineering  experiments  can  be  integrated  into  either  ground-based  or  air¬ 
borne  flight  trainers,  but  new  techniques  will  have  to  be  developed  for  their 
routine  use  in  pilot  training,  certification,  and  currency  assurance,, 
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Computer  Graphics 

The  simulation  of  extracockpit  visual  cues  is  essential  for  training  in 
ground-referenced  maneuvers  involving  great  danger  in  actual  flight.  Extremely 
costly  visual  systems  have  been  employed  for  training  in  high-speed,  low-altitude 
military  operations  and  in  emergency  procedures,  such  as  single-engine  approaches 
and  engine  loss  on  takeoff  in  multiengine  transpoi!  aircraft.  High  costs  are 
justified  in  such  cases.  However,  there  is  an  urgent  need  for  less  costly  but  none¬ 
theless  effective  visual  systems  for  use  in  various  phases  of  flight  training.  Perhaps 
the  greatest  payoff  would  be  found  in  the  initial  training  of  pilots  to  land  an  air¬ 
plane  safely  with  a  minimum  of  exposure  to  the  hazards  of  presolo  and  early  post- 
solo  landing  practice. 

Valverde  (1968)  emphasizes  the  importance  of  understanding  the 
capabilities  and  limitations  of  visual  equipment  in  order  to  evaluate  properly  its 
use  to  meet  specific  training  requirements.  He  points  out,  for  example,  that  a 
large  generator  is  necessitated  by  the  use  of  a  large  visual  envelope.  Therefore, 
if  a  small  envelope  can  be  used,  the  cost  saving  will  be  extended  to  other  equip¬ 
ment  dependent  on  it. 

A  computer-generated  line-drawing  display  system  (LDS-1)  developed  by 
the  Evans  and  Sutherland  Computer  Corporation  (Ogden,  1970)  fits  into  Valverde's 
small  envelope  category.  This  graphic  display  system  allows  automatic  windowing 
and  perspective  projection  of  three-dimensional  objects,  such  as  an  aircraft  carrier 
or  an  airport  with  runways  and  hangars,  and  therefore  lends  itself  to  the  simulation 
of  approaches  to  landings  and  other  contact  flight  operations  requiring  a  limited 
field  of  view. 

The  Advanced  Simulator  for  Undergraduate  Pilot  Training  (ASUPT)  being 
developed  for  the  Flying  Training  Division  of  the  USAF  Human  Resources  Laboratory 
presents  an  enormous  computer-generated  visual  envelope  around  the  simulated 
cockpit  of  a  T-37  airplane  (Gum,  Knoop,  Basinger,  Guterman,  and  Foley,  1972; 
Smith,  1972).  This  application  of  computer  graphics  presents  a  somewhat  less  than 
literal  black  and  white  image  of  the  outside  world  on  seven  36-inch  circular  CRTs, 
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each  framed  by  a  pentagonal  display  window,  in  a  faceted  arrangement  covering  the 
full  forward,  lateral,  and  vertical  limits  of  the  external  visual  field  from  the  cock¬ 
pit  of  the  T-37.  This  colossal  device  is  designed  to  allow  the  systematic  experimen¬ 
tal  determination  of  the  external  visual  cues  contributing  significantly  to  contact 
flight  training. 

Another  advanced  application  of  computer  graphic  techniques,  developed 
jointly  by  Hughes  Aircraft  Company  and  the  University  of  Illinois  for  the  Federal 
Aviation  Administration,  generates  a  moving-map  display  for  cockpit  presentation, 
continuously  showing  present  position,  heading,  and  area  navigation  guidance 
commands.  Similar  systems  have  been  developed  by  several  companies  including 
Boeing,  Astronautics,  and  Sperry-Phoenix. 

Incremental  Transfer  Effectiveness 

To  determine  the  relative  value  of  simulator  training,  Roscoe  (1971;  1972) 
proposed  the  concept  of  "Incremental  Transfer  Effectiveness"  which  postulates  a 
function  found  by  comparing  successive  increments  of  time  spent  in  one  training 
task  with  successive  increments  of  time  saved  in  subsequent  training.  When  the 
Incremental  Transfer  Effectiveness  Ratio  drops  below  the  ratio  of  the  hourly  cost  for 
ground  trainers  to  that  of  training  aircraft,  continued  ground  framing  is  not  cost 
effective. 

The  Incremental  Transfer  Effectiveness  concept  recognizes  the  decreasing 
value  of  successive  increments  of  simulator  training  in  terms  of  the  time  saved  in 
generally  more  expensive  equipment.  Povenmire  and  Roscoe  (1973)  demonstrated 
the  negatively  decelerated  relationship  between  hours  saved  in  the  Cherokee  air¬ 
plane  and  hours  spent  in  the  Link  GAT-1  in  the  training  of  a  Private  Pilot.  Com¬ 
parison  of  the  Incremental  Transfer  Effectiveness  Functions  of  different  training 
provides  a  rational  basis  for  procurement  and  use  in  economic  terms. 

Response  Surface  Methodology 

Previous  research  has  concentratsd  on  the  separate  effects  of  numerous 
variables  important  in  simulator  training,  but  little  effort  has  been  directed  toward 
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investigating  the  simultaneous  effects  of  these  variables.  It  is  possible  that 
important  interactions  may  be  present  or  that  the  effect  of  one  variable  may  be  so 
strong  that  it  overrides  other  variables.  Methodologically,  however,  it  is 
extremely  difficult  to  examine  many  variables  at  once  without  quickly  approaching 
an  unwieldy  number  of  essential  data  points.  For  example,  if  three  variables  were 
observed  at  three  levels  in  a  traditional  factorial  analysis  of  variance  design,  27 
treatment  combinations  or  data  points  would  be  required  for  each  replication  of  the 
design.  If  seven  variables  were  investigated  at  three  levels  each,  2187  data  points 
would  be  required  for  each  replication.  Obviously,  the  latter  experiment  would 
not  be  conducted.  It  is  also  not  surprising  that  such  a  methodological  impass  was 
quickly  realized  in  early  research  on  flight  simulators  (Williams  and  Adelson, 

1954). 

Research  techniques  called  Response  Surface  Methodology  (RSM)  have  been 
developed  for  investigating  many  variables  simultaneously.  Box  and  Wilson  (1951) 
originally  used  RSM  to  determine  the  optimum  combination  of  variables  for 
producing  the  maximum  yield  of  a  chemical  reaction.  The  RSM  designs  minimize 
the  number  of  data  collection  points  necessary  to  determine  a  multiple  regression 
prediction  equation  describing  the  relationship  between  a  predicted  score  and  the 
experimental  variables.  Details  and  examples  of  this  technique  are  provided  by 
Box  and  Hunter  (1957)  and  Cochran  and  Cox  (1957). 

Recently,  Williges  and  Simon  (1971)  discussed  the  utility  of  using  RSM 
techniques  in  human  performance  research.  In  addition  to  the  economy  of  the  data 
collection,  the  designs  are  Flexible  and  efficient.  The  designs  are  flexible  in  that 
the  data  can  be  collected  in  sequential  order.  At  the  end  of  each  stage  of  data 
collection,  the  experimenter  can  analyze  his  results  and  decide  on  the  appropriate 
data  points  to  investigate  during  the  next  stage  of  experimentation.  The  designs 
are  also  efficient  in  that  controls  are  readily  available  for  undesirable  fluctuations 
when  the  experiment  is  extended  over  time.  However,  certain  design  modifica¬ 
tions  are  necessary  before  these  techniques  can  be  used  successfully  to  assess  human 
behavior.  Some  of  these  considerations  are  described  by  Clark  and  Williges  (1972). 
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With  the  increased  us2  of  RSM  in  engineering  research,  it  is  surprising  that 
limited  applications  have  been  made  to  behavioral  research.  Only  two  studies 
concerned  with  problems  of  human  learning  have  used  RSM.  Meyer  (1963)  used 
RSM  to  study  the  effects  o;  degree  of  original  learning,  time  between  interpolated 
and  original  learning,  length  of  the  interpolated  list,  and  degree  of  interpolated 
learning  on  the  amount  of  retroactive  inhibition  in  verbal  learning.  He  plotted  a 
response  surface  relating  the  four  independent  variables  to  amount  of  recall. 
Williges  and  Baron  (1972)  used  RSM  to  plot  a  transfer  surface  of  trials  to  criterion 
in  an  epicycloid  purscit  rotor  task  as  a  function  of  tracking  speed  during  training, 
time  between  training  trials,  and  number  of  training  trials  on  a  simple  pursuit  rotor 
task. 

The  RSM  technique  appears  to  be  a  viable  procedure  or  model  for  systemat¬ 
ically  developing  a  training  simulator.  First,  it  allows  for  simultaneous  investiga¬ 
tion  of  many  varicbles.  Second,  the  sequential  research  strategy  of  RSM  provides 
an  orderly  procedure  for  determining  the  variables  of  importance  in  simulation  to 
maximize  learning,  transfer,  and  retention.  Third,  the  resulting  prediction  equa¬ 
tions  can  be  used  to  determine  tradeoffs  among  the  various  independent  variables 
important  in  simulation  to  maintain  a  specific  level  of  learning,  transfer,and 
retention.  Finally,  the  separate  RSM  prediction  equations  for  level  of  learning, 
transfer,  and  retention  can  be  compared  to  determine  the  necessary  tradeoffs  among 
the  important  simulation  variables  to  optimize  systematically  the  combined  level  of 
learning,  transfer,  and  retention  provided  by  a  particular  simulator. 

One  overall  limitation  of  research  on  training  simulators  appears  to  be  that 
simple  piecemeal  approaches  are  used  to  solve  complex  research  problems.  The 
potential  power  of  RSM  is  that  it  allows  the  investigator  to  examine  the  problems 
of  simulation  research  from  a  complex,  multiparameter,  yet  systematic,  point  of 
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PROSPECTUS 

User  demand  for  air  transportation,  recreational  flying,  and  an  ever- 
increasing  variety  of  agricultural ,  industrial,  and  sciertific  flight  operations  is 
placing  unprecedented  pressures  on  the  National  Airspace  System  (NAS).  The 
rapidly  increasing  complexity  of  the  system  itself  is  demanding  new  levels  of  flying 
skill  and  knowledge  to  which  few  pilots  have  been  trained,  and  training  costs  to 
prepare  pilots  to  operate  safely  and  effectively  in  the  NAS  are  becoming  prohibi¬ 
tive.  Furthermore,  there  is  inadequate  assurance  that  those  presently  flying  are 
qualified  to  do  so,  and  this  problem  is  growing. 

What  is  needed  is  a  scientifically  rigorous  investigation  into  fundamental 
flight  requirements,  including  not  only  the  perceptual,  cognitive,  and  motor  skills 
required  of  pilots,  but  also  the  attitudes  and  judgmental  factors  essenticl  to  safe 
flight.  The  investigation  must  start  with  the  identification  of  the  types  of  flight 
operations,  or  missions,  that  will  be  undertaken  during  tne  forseeable  future  and 
the  functions  to  be  performed  by  pilots  in  such  operations.  From  this  functional 
analysis  must  be  derived  the  minimum  standards  of  skill,  knowledge,  and  judgment 
required  of  all  categories  of  pilots  permitted  to  fly  in  the  National  Airspace  System. 
Current  pilot  training  and  certification  practices  must  be  evaluated  in  this  new 
context.  Where  existing  requirements  and  methods  are  found  to  be  deficient,  new 
approaches  must  be  devised  to  close  the  training  and  certification  gaps  at  a 
bearable  cost. 

A  new  pilot  training,  certification,  and  currency  assurance  system  is 
needed,  one  that  will  automatically  qualify  each  pilot  for  his  particular  level  of 
operation  at  a  bearable  cost  to  him  as  well  as  to  the  aviation  community. 
Representative  advances  in  training  technology  applicable  to  this  objective  include 
computer-aided  cognitive  training  and  testing,  automatically  adaptive  skill 
training  and  performance  assessment,  and  the  extended  use  of  simulation  to 
previously  unexploited  areas  of  pilot  training,  certification,  and  currency 
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